“Software Testing Applications,” such as IBM's Rational Robot as well as the latest product, Functional Tester (RFT) have been known for some time for quality testing software targets, such as web or server applications, stand-alone computing applications, web services, computer program products, GUI interfaces, etc., to name a few. During use, test cases of the software target are written as to examine whether features of the target are operating properly. For example, it may be the situation that a web service providing stock market quotes with a GUI front end has need of determining whether a user fill-in box for stock symbols actually produces a stock quote for the symbol upon submission. In such instance, the software testing application uses code for an individual test case to determine whether the web service works as advertised. Regardless of success, it is important to the user of the software testing application to know and record the results of the test, e.g., whether the test passed, failed, met a predetermined metric, or other. (Of course, an infinite number of test cases per software targets are possible and the foregoing is merely a contextual illustration that skilled artisans will readily understand.)
One well known and open source testing framework, known commonly as JUnit, is another tool in the quality assurance of testing software targets. However, constructing test cases that integrate well with the JUnit framework and automatically generating HTML reports based upon test results can sometimes be cumbersome when using third party test tools, e.g., RFT. As is known, test case construction may involve many classes and methods in a language such as C, C++, or Java and problems exist when trying to integrate the JUnit test case execution followed by a user report generation in an automated fashion.
Another problem with using the JUnit framework to execute test cases requires extending the TestCase class by design. In turn, this may lead to programming issues when trying to modifying a third party test tool class hierarchy. For instance, the third party test tool's class loader may generate, or “throw,” exceptions when trying to load the altered class hierarchy.
Still another problem lies with report generation in a format useful for users. Namely, JUnit typically generates reports in an XML format, which is non-visually recognized by users. Converting the XML to a visually useful format, such as HTML, requires several transformation classes. While this may be typically done using an Apache Ant script, which runs as a separate process, launching the Ant script requires still further programming code. In other words, it is not self-executing. Interoperability issues may also exist between Linux- and Windows-brand operating systems, for example.
Accordingly, there is need in the art of software testing applications to easily and conveniently generate user reports that are presented in useful format. There is further need to make the reporting “invisible” relative to the users and software testing applications to minimize inconvenience. It is also important to generate reports quickly and provide appropriate results. In that many users already own and/or use a software testing application, it is further desirable to retrofit existing applications to avoid the costs of providing wholly new products. Taking advantage of existing testing frameworks, such as JUnit, and transformation-of-languages programs, such as Ant, is another feature that would optimizes existing resources. Naturally, any improvements along such lines should further contemplate good engineering practices, such as relative inexpensiveness, stability, ease of implementation, low complexity, flexibility, etc.